The present invention relates to a method and an apparatus for processing image data, and more particularly, to an image data processing method which generates color difference data from complementary color data representing complementary color components.
FIG. 1 is a block diagram showing the configuration of a conventional image sensing device which uses a CCD image sensor 1, and FIG. 2 is a plan view showing an example of a conventional mosaic color filter attached to the CCD image sensor 1.
The CCD image sensor 1 has a plurality of light-receiving pixels, a plurality of vertical shift registers and usually a horizontal shift register. The light-receiving pixels are arranged in a matrix form on a light-receiving surface at regular intervals and produce and store information charges corresponding to the image of a sensed object. The vertical shift registers are arranged to correspond to the columns of the light-receiving pixels and sequentially shift the information charges stored in the light-receiving pixels in the vertical direction. The horizontal shift register is arranged on the output side of the vertical shift registers and receives the information charges output from the vertical shift registers, and then transfers the information charges row by row. This allows the horizontal shift register to output an image signal I0 in accordance with the information charges stored in the light-receiving pixels.
An analog processing circuit 2 performs a process, such as sampling and holding or level clamping, on the image signal I0 input from the CCD image sensor 1 to produce an image signal I1 which conforms to a predetermined format. For example, in the sample and hold process, only an image signal having a certain-signal level is extracted from the image signal I0 having reset levels and signal levels which are alternately repeated in synchronism with the output operation of the CCD image sensor 1. In the level clamping process, the black reference level set at the end of the horizontal scanning period of the image signal I0 is clamped to a predetermined level every horizontal scanning period. An A/D converter circuit 3 quantizes the image signal I1 received from the analog processing circuit 2 to generate image data D which represents the information with a digital value corresponding to each light-receiving pixel of the CCD image sensor 1.
A digital processing circuit 4 performs a process, such as color distribution or a matrix operation, on the image data D received from the A/D converter circuit 3 and generates luminance data Y and color difference data U and V. For example, in the color distribution process, the image data D is distributed in accordance with the color arrangement of a color filter attached to the light-receiving surface of the CCD image sensor 1, generating a plurality of color component data. Further, in the matrix operation process, primary color data corresponding to the three primary colors of light are generated from the individually distributed color component data, and are then combined at a predetermined ratio, thereby generating the color difference data U, V.
A driver 5 responds to various timing signals from a timing control circuit 6 and supplies a multi-phase drive clock to the shift registers of the CCD image sensor 1. For example, a 4-phase vertical transfer clock xcfx86v is supplied to the vertical shift registers, and a 2-phase horizontal transfer clock xcfx86h is supplied to the horizontal shift register. In accordance with a reference clock having a predetermined cycle, the timing control circuit 6 produces a vertical timing signal VT, which determines the vertical scan timing of the CCD image sensor 1, and a horizontal timing signal HT, which determines the horizontal scan timing, and supplies the timing signals to the driver 5. At the same time, the timing control circuit 6 supplies a timing clock CT to the analog processing circuit 2, the A/D converter circuit 3, and the digital processing circuit 4 in order to synchronize the operations of the circuits 2, 3, 4 with the output operation of the CCD image sensor 1.
When performing color image sensing, a color filter for color distribution is attached to the light-receiving surface in order to associate the individual light-receiving pixels of the CCD 1 with predetermined color components. A stripe type filter having a plurality of segments connected in the vertical direction or a mosaic type filter having a plurality of segments associated with the light-receiving pixels may be used as the color filter. For example, the mosaic type color filter, shown in FIG. 2, is split into a plurality of segments corresponding to each pixel of the light-receiving section of the CCD image sensor 1 and color components of Ye (yellow), Cy (cyan), W (white) and G (green) are cyclically assigned to each segment. The W and G components are alternately arranged in odd rows and the Ye and Cy components are alternately arranged in even rows. In an image signal obtained from the CCD image sensor 1, to which such color filter is attached, the W and G components are repeated when reading odd rows and the Ye and Cy components are repeated when reading even rows.
FIG. 3 is a block diagram showing the configuration of the digital signal processing circuit 4, and FIG. 4 is a timing diagram for describing the operation of the processing circuit 4. FIG. 4 corresponds to the case where the mosaic type color filter shown in FIG. 2 is attached to the CCD image sensor 1.
A color distribution circuit 11 separates the image data D in which each color component continues in the arrangement order of the segments of the color filter. The distribution circuit 11 then generates color component data C[Ye], C[Cy], C[G] and C[W]. For the image data D input from the A/D converter circuit 3, as shown in FIG. 4, the G and W components continue alternately when reading odd rows (ODD) and the Ye and Cy components continue alternately when reading even rows (EVEN). Accordingly, the color distribution circuit 11 retains at least one row of the image data D to allow the output of all the color component data C[Ye], C[Cy], C[G] and C[W] at the time of reading each row. Specifically, during reading of an odd row, the color distribution circuit 11 separates the image data D for the odd row and outputs the color component data C[G] and C[W]. At the same time, the color distribution circuit 11 separates the image data D for the previously read even row and outputs the color component data C[Ye] and C[Cy]. Further, this scheme causes the color component data C[Ye], C[Cy], C[G] and C[W] to be output intermittently when the image data D is output serially. The intermittent portions of the color component data are then interpolated by outputting the same color component data twice in succession.
A color calculation circuit 12 performs a color computation process according to, for example, the following equations on the color component data C[Ye], C[Cy], C[G] and C[W] input from the color distribution circuit 11, to generate primary color data P[R], P[G] and P[B] corresponding to the three primary colors (R: red, G: green and B: blue) of light.
Yexe2x88x92G=R
Cyxe2x88x92G=B
G=G
A white balance control circuit 13 assigns specific gains to each of the primary color data P[R], P[G] and P[B] input from the color calculation circuit 12 to adjust the balance of each color. In other words, in the white balance control circuit 13, because this adjustment compensates for differences in the sensitivities of the light-receiving pixels of the CCD image sensor 1 which depend on each color component, the gains of the primary color data P[R], P[G] and P[B] are individually set to improve the color reproduction of a reproduced image.
A color difference matrix circuit 14 generates the color difference data U and V from the primary color data P[R], P[G] and P[B] input from the white balance control circuit 13. The color difference matrix circuit 14 combines the respective primary color data P[R], P[G] and P[B] at a ratio of 3:6:1 to generate luminance information. Then, the color difference matrix circuit 14 subtracts the luminance information from the primary color data P[B] corresponding to the B component to generate the color difference data U. Further, the color difference matrix circuit 14 subtracts the luminance information from the primary color data P[R] corresponding to the R component to generate the color difference data V.
A luminance calculation circuit 15 combines the four color components included in the image data D provided to the color distribution circuit 11 to generate the luminance data Y. That is, if each of the components Ye, Cy, G, W are combined, then                               Ye          +          Cy          +          G          +          W                =                              (                          B              +              G                        )                    +                      (                          R              +              G                        )                    +          G          +                      (                          R              +              G              +              B                        )                                                  =                              2            ⁢            R                    +                      4            ⁢            G                    +                      2            ⁢            B                              
This allows luminance data in which the R, G and B components are combined at a ratio of 1:2:1 to be obtained. While the NTSC standards define a luminance signal produced by combining the R, G and B components at a ratio of 3:6:1, a luminance signal produced by combining the components at a ratio close to this ratio does not cause a practical problem.
An aperture circuit 16 enhances a specific frequency component included in the luminance data to generate aperture data, and adds the aperture data to the luminance data to generate a luminance data signal Y. In other words, to enhance the outline of the image of a sensed object, the aperture circuit 16 performs a filtering process on the image data D to generate aperture data such that the frequency component of one fourth the sampling frequency, which is used to obtain the image data D from the image signal, is enhanced. The luminance data signal Y generated in this manner is supplied to an external display device or recording device together with the color difference data U and V.
Because the R and B components are generated in the color computation process by the color calculation circuit 12 by subtracting the G component from the Ye and Cy components, respectively, the R or B component may show a negative value because of the unevenness of spectral characteristics of the color filter. For example, for light in which the G component is strong and the R or B component are weak, the Ye or Cy component has a slightly higher value than the G component, and the R or B component show a positive value close to xe2x80x9c0xe2x80x9d. However, if the desired light cannot transmitted through the Ye or Cy and the Ye or Cy component can be obtained only by a lower value than the G component, the R or B component has a negative value as a result of the color computation process. Such negative values are not original color components, and thus produce false signals, thereby deteriorating the image quality of the reproduced image.
Accordingly, it is an object of the present invention to provide an image data processing apparatus that is substantially unaffected by the color computation process even if the spectral sensitivity of each color component in the color filter is not the same.
To achieve the above object, the present invention provides a method for generating a first color difference data and a second color difference data by processing a first complementary color data, a second complementary color data, and a third complementary color data, each representing a respective complementary color of one of the three primary colors. The method includes the steps of multiplying the first complementary color data and the second complementary color data to generate a first product, multiplying the first complementary color data and the third complementary color data to generate a second product, multiplying the second complementary color data and the third complementary color data to generate a third product, subtracting the second product from the first product to generate a first difference, subtracting the second product from the third product to generate a second difference, extracting the square root of an absolute value of the first difference to generate a first root, extracting the square root of an absolute value of the second difference to generate a second root, adding a polarity indication code of the first difference to the first root, adding a polarity indication code of the second difference to the second root, and synthesizing the first and second roots, to which the polarity indication codes have been added, to generate the first and second color difference data.
A further aspect of the present invention provides an apparatus for generating a first color difference data and a second color difference data by processing image data including a first complementary color data, a second complementary color data, and a third complementary color data, each representing a respective complementary color of one of the three primary colors. The apparatus includes a distribution circuit for distributing the image data into the first complementary color data, the second complementary color data, and the third complementary color data. A multiplication circuit multiplies the first complementary color data and the second complementary color data to generate a first product, the first complementary color data and the third complementary color data to generate a second product, and the second complementary color data and the third complementary color data to generate a third product. A subtraction circuit subtracts the second product from the first product to generate a first difference, and the second product from the third product to generate a second difference. A square root extraction circuit extracts a square root of an absolute value of the first difference to generate a first root, and a square root of an absolute value of the second difference to generate a second root. A polarity adding circuit adds a polarity indicating code of the first difference to the first root, and a polarity indicating code of the second difference to the second root to generate first and second polarized roots. A color difference matrix circuit synthesizes the first and second polarized roots to generate the first and second color difference data.
Another aspect of the present invention provides an apparatus for generating first and second color difference data by processing image data including a first complementary color data, a second complementary color data, and a third complementary color data, each of the first to third complementary color data representing a respective complementary color of the three primary colors of light. The apparatus includes a distribution circuit for distributing the image data into the first complementary color data, the second complementary color data, and the third complementary color data. A color balance control circuit is connected to the distribution circuit to apply a gain to each of the first to third complementary color data and generate first to third gain adjusted complementary color data. A multiplication circuit is connected to the color balance control circuit to multiply the first gain adjusted complementary color data and the second gain adjusted complementary color data to generate a first product, the first gain adjusted complementary color data and the third gain adjusted complementary color data to generate a second product, and the second gain adjusted complementary color data and the third gain adjusted complementary color data to generate a third product. A subtraction circuit subtracts the second product from the first product to generate a first difference, and the second product from the third product to generate a second difference. A square root extraction circuit extracts a square root of an absolute value of the first difference to generate a first root, and a square root of an absolute value of the second difference to generate a second root. A polarity adding circuit adds a polarity indicating code of the first difference to the first root, and a polarity indicating code of the second difference to the second root to generate first and second polarized roots. A color difference matrix circuit synthesizes the first and second polarized roots to generate the first and second color difference data.
Other aspects and advantages of the present invention will become apparent from the following description, taken in conjunction with the accompanying drawings, illustrating by way of example the principles of the invention.